Multi-sensory emoticons in a communication system

ABSTRACT

Methods and systems for providing dynamic emoticons during chat sessions across game consoles are disclosed. A dynamic emoticon may include haptic, visual, and/or auditory components. Each component may have an associated intensity level, e.g., based on a pressure with which a user of the game console presses a controller button or actuates a control input. The emoticon&#39;s auditory component may include or identify a sound effects filter or a known audio file; the emoticon&#39;s visual component may include or identify a visual effects filter or a known video or graphic file; the emoticon&#39;s haptic component may cause vibration of or force feedback on the controller. Emoticons may be sent from one voice chat participant&#39;s game console to another voice chat participant&#39;s game console across a network such as the Internet.

This application is a non-provisional application of and claims priorityto Provisional Application Ser. No. 60/569,614, filed May 11, 2004,entitled “Dynamic Emoticons.”

FIELD OF THE INVENTION

The invention relates generally to computer games, video games, and gameconsoles. More specifically, the invention provides a method ofproviding a voice chat service on a main control application on a videogame console when no game is otherwise being played by a user of thegame console.

BACKGROUND OF THE INVENTION

Computer games, video games, arcade games, and other forms of electronicgames have advanced from simple games such as Pong, Breakout, Asteroids,and Space Invaders to complex, multiplayer, online high-resolution gamessuch as CRIMSON SKIES®, published by Microsoft Game Studios of RedmondWash.

Initial online multiplayer games had no mechanism for players tocommunicate with each other while playing the game, unless the playerswere either physically located near each other or communicated through aseparate medium such as a separate telephone call. As online gamingprogressed, online games began providing players limited communicationcapabilities, such as being able to text chat while playing the game.

With the launch of XBOX® LIVE by Microsoft Corporation of Redmond, Wash.in November 2002 came in-game voice chat capabilities. Microsoftprovides a software development kit (SDK) for XBOX® LIVE with which gamedevelopers can include voice chat support in their game titles. In orderfor two players to chat, they must have the same game title executing ontheir respective game consoles, and both be signed in to the XBOX® LIVEonline gaming environment.

When two players do not have the same game title executing on theirrespective XBOX® game consoles, they cannot voice chat with each other.Instead, one must send the other an invitation for a specific game, andthe recipient of the invitation must change game media to the specificgame. Even then, players must typically match up to play a game media tothe specific game. Even then, players must typically match up to play agame before they will be able to voice chat with one another. If the twoplayers subsequently decide to play a different game, then the playersmust change media again to be able to continue voice chatting, makinggame coordination lengthy and difficult.

Another problem that often occurs is the inability of a player toadequately respond to a game invitation beyond simply selecting not tojoin the Friend. The invited gamer might not currently wish to play ormay want to but lack the specified game disc. This would result ingarners either not matching up to play or spending excess time invitingeach other back and forth until stumbling upon a game that both wereable to play.

Thus it would be an advancement in the art to provide a mechanism forgarners to connect and carry on real-time voice communications withoutrequiring that each gamer have the same game media loaded in theirrespective game consoles, thus creating an opportunity for expandedcommunity, building relationships, and allowing garners the opportunityto “meet up” with Friends online in order to talk and plan out gametitle selections and game times.

Regardless of the environment in which the chat sessions occur, however,present chat applications provide only limited mechanisms by whichplayers can communicate. That is, in a text chat, players can only typeback and forth, with limited expression in the form of emoticons. Basicemoticons are include the following: :-) or :) Basic smiling face; usedfor humor and sometimes sarcasm :-( or :( Basic frowning face; used forsadness or anger ;-) or ;) Half-smiling or winking face; more often usedfor sarcasm :-/ Wry face; used for wry humor

Similarly, in a voice chat, players can only speak to each other,without the advantage of any emoticons because there is no text involvedin a voice chat. Text chat systems and electronic mail have enabledusers only to send text-based emoticons, but no system has created atwo-way real-time voice chat capability or allowed user to not only sendiconic emoticons but to also send sound alerts and to affect the actualhardware of the other user's system. Thus, it would be a furtheradvancement in the art to provide new mechanisms of communication in avoice, audio, and/or video chat session that allow interacting users toconvey emotions, actions, and the like.

BRIEF SUMMARY OF THE INVENTION

The following presents a simplified summary of the invention in order toprovide a basic understanding of some aspects of the invention. Thissummary is not an extensive overview of the invention. It is notintended to identify key or critical elements of the invention or todelineate the scope of the invention. The following summary merelypresents some concepts of the invention in a simplified form as aprelude to the more detailed description provided below.

To overcome limitations in the prior art described above, and toovercome other limitations that will be apparent upon reading andunderstanding the present specification, the present invention isdirected to dynamic emoticons in a chat session across data processingsystems capable of executing entertainment applications.

According to a first aspect of the invention, an emoticon may include ahaptic component, e.g., causing vibration or force feedback to occur ona game controller.

According to another aspect of the invention, an emoticon may include anintensity level, e.g., defining an intensity of a haptic or visualcomponent, or to determine the volume of an auditory component.

According to another aspect of the invention, an emoticon may be definedby a custom script created by a participant in a chat session. Thescript may define emoticon elements (e.g., haptic, visual, and auditory)and may further define element actions.

According to another aspect of the invention, an emoticon may include anauditory component identifying or defining an audio file for playback,or a sound effects filter for altering audio otherwise output by thedata processing system

According to another aspect of the invention, a gaming system having adata processing unit, a controller, a network port, and a memory mayprovide a dynamic emoticon functionality. The gaming system executesentertainment applications from a media drive, and executes the voicechat application from a memory. The controller comprises a plurality ofcontrol inputs (e.g., buttons, joysticks, triggers, etc.) for providinginput to the gaming system. The network port allows communication withother gaming systems over a network such as the Internet. The residentmemory stores the voice chat application as computer executable code forgenerating an emoticon message based on user input received from thecontroller.

Aspects of the invention also provide methods for sending and receivingdynamic emoticon messages.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete understanding of the present invention and theadvantages thereof may be acquired by referring to the followingdescription in consideration of the accompanying drawings, in which likereference numbers indicate like features, and wherein:

FIG. 1 illustrates a gaming system that may be used according to anillustrative embodiment of the invention.

FIG. 2 illustrates a block diagram of the gaming system shown in FIG. 1.

FIG. 3 illustrates a block diagram of a network gaming system accordingto an illustrative embodiment of the invention.

FIG. 4 illustrates another block diagram of an online gaming environmentaccording to an illustrative embodiment of the invention.

FIG. 5 illustrates a screenshot of an online gaming environment menuscreen according to an illustrative embodiment of the invention.

FIG. 6 illustrates a screenshot of a voice chat menu screen according toan illustrative embodiment of the invention.

FIG. 7 illustrates a screenshot of an option screen for a user on aFriends list according to an illustrative embodiment of the invention.

FIG. 8 illustrates a method for administering a voice chat according toan illustrative embodiment of the invention.

FIG. 9 illustrates a screenshot upon selection of a user in a voice chataccording to an illustrative embodiment of the invention.

FIG. 10 illustrates a screenshot of a submenu upon selection of theemoticon option in FIG. 9 according to an illustrative embodiment of theinvention.

FIG. 11 illustrates an alternative screenshot of a submenu uponselection of the emoticon option in FIG. 9 according to anotherillustrative embodiment of the invention.

FIG. 12 illustrates a method for sending a dynamic emoticon in a voicechat according to an illustrative embodiment of the invention.

FIG. 13 illustrates an emoticon instruction message according to anillustrative embodiment of the invention.

FIG. 14 illustrates an emoticon message according to anotherillustrative embodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

In the following description of the various embodiments, reference ismade to the accompanying drawings, which form a part hereof, and inwhich is shown by way of illustration various embodiments in which theinvention may be practiced. It is to be understood that otherembodiments may be utilized and structural and functional modificationsmay be made without departing from the scope of the present invention.

FIG. 1 illustrates an example of a suitable gaming system environment100 on which computer games, video games, and or other electronic games(collectively referred to herein as computer games) may be played. Thegaming system environment 100 is only one example of a suitablecomputing environment and is not intended to suggest any limitation asto the scope of use or functionality of the invention. Neither shouldthe gaming system environment 100 be interpreted as having anydependency or requirement relating to any one or combination ofcomponents illustrated in the illustrative operating gaming systemenvironment 100.

The invention is operational with numerous other general purpose orspecial purpose computing system environments or configurations.Examples of well known computing systems, environments, and/orconfigurations that may be suitable for use with the invention include,but are not limited to, personal computers; server computers; portableand hand-held devices such as personal digital assistants (PDAs), tabletPCs or laptop PCs; multiprocessor systems; microprocessor-based systems;set top boxes; programmable consumer electronics; network PCs;minicomputers; mainframe computers; electronic game consoles,distributed computing environments that include any of the above systemsor devices; and the like.

The invention may be described in the general context ofcomputer-executable instructions, such as program modules, beingexecuted by a computer. Generally, program modules include routines,programs, objects, components, data structures, etc. that performparticular tasks or implement particular abstract data types. Theinvention may also be practiced in distributed computing environmentswhere tasks are performed by remote processing devices that are linkedthrough a communications network. In a distributed computingenvironment, program modules may be located in both local and remotecomputer storage media including memory storage devices.

FIG. 1 shows an exemplary gaming system 100. Gaming system 100 mayinclude a game console 102 and up to four controllers, as represented bycontrollers 104(1) and 104(2). The game console 102 is equipped with aninternal hard disk drive and a portable media drive 106 that supportsvarious forms of portable storage media as represented by opticalstorage disc 108. Examples of suitable portable storage media includeDVD, CD-ROM, game discs, and so forth.

Game console 102 has four slots 110 on its front face to support up tofour controllers, although the number and arrangement of slots may bemodified. A power button 112 and an eject button 114 are also positionedon the front face of the game console 102. The power button 112 switchespower to the game console and the eject button 114 alternately opens andcloses a tray of the portable media drive 106 to allow insertion andextraction of the storage disc 108.

Game console 102 may connect to a television or other display (notshown) via A/V interfacing cables 120. A power cable 122 provides powerto the game console. Game console 102 may further be configured withbroadband network capabilities, as represented by the cable or modemconnector 124 to facilitate access to a network, such as the Internet.

Each controller 104 may be coupled to the game console 102 via a wire orwireless interface. In the illustrated implementation, the controllersare USB (Universal Serial Bus) compatible and are connected to theconsole 102 via USB cables 130. Controller 102 may be equipped with anyof a wide variety of user interaction mechanisms. As illustrated in FIG.1, each controller 104 is equipped with two thumbsticks 132(1) and132(2), a D-pad 134, buttons 136 (e.g., ‘A’, ‘B’, ‘X’, ‘Y’), and twotriggers 138. These mechanisms are merely representative, and otherknown gaming mechanisms may be substituted for or added to those shownin FIG. 1.

A memory unit (MU) 140 may be inserted into the controller 104 toprovide additional and portable storage. Portable memory units enableusers to store game parameters and user accounts, and port them for playon other consoles. In the described implementation, each controller isconfigured to accommodate two memory units 140, although more or lessthan two units may be employed in other implementations. A headset 142may be connected to the controller 104 or game console 102 to provideaudio communication capabilities. Headset 142 may include a microphonefor audio input and one or more speakers for audio output.

Gaming system 100 is capable of playing, for example, games, music, andvideos. With the different storage offerings, titles can be played fromthe hard disk drive or the portable medium 108 in drive 106, from anonline source, or from a memory unit 140. For security, in someembodiments executable code can only be run from the portable medium108. A sample of what gaming system 100 is capable of playing includegame titles played from CD and DVD discs, from the hard disk drive, orfrom an online source; digital music played from a CD in the portablemedia drive 106, from a file on the hard disk drive (e.g., Windows MediaAudio (WMA) format), or from online streaming sources; and digitalaudio/video played from a DVD disc in the portable media drive 106, froma file on the hard disk drive (e.g., Active Streaming Format), or fromonline streaming sources.

FIG. 2 shows functional components of the gaming system 100 in moredetail. The game console 102 has a central processing unit (CPU) 200 anda memory controller 202 that facilitates processor access to varioustypes of memory, including a flash ROM (Read Only Memory) 204, a RAM(Random Access Memory) 206, a hard disk drive 208, and the portablemedia drive 106. The CPU 200 is equipped with a level 1 cache 210 and alevel 2 cache 212 to temporarily store data and hence reduce the numberof memory access cycles, thereby improving processing speed andthroughput.

The CPU 200, memory controller 202, and various memory devices areinterconnected via one or more buses, including serial and parallelbuses, a memory bus, a peripheral bus, and a processor or local bususing any of a variety of bus architectures. By way of example, sucharchitectures can include an Industry Standard Architecture (ISA) bus, aMicro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, aVideo Electronics Standards Association (VESA) local bus, and aPeripheral Component Interconnects (PCI) bus also known as a Mezzaninebus.

As one suitable implementation, the CPU 200, memory controller 202, ROM204, and RAM 206 are integrated onto a common module 214. In thisimplementation, ROM 204 is configured as a flash ROM that is connectedto the memory controller 202 and a ROM bus (not shown). RAM 206 isconfigured as multiple DDR SDRAM (Double Data Rate Synchronous DynamicRAM) that are independently controlled by the memory controller 202 viaseparate buses (not shown). The hard disk drive 208 and portable mediadrive 106 are connected to the memory controller via the PCI bus and anATA (AT Attachment) bus 216.

A 3D graphics processing unit 220 and a video encoder 222 form a videoprocessing pipeline for high speed and high resolution graphicsprocessing. Data is carried from the graphics processing unit 220 to thevideo encoder 222 via a digital video bus (not shown). An audioprocessing unit 224 and an audio codec (coder/decoder) 226 form acorresponding audio processing pipeline with high fidelity and stereoprocessing. Audio data is carried between the audio processing unit 224and the audio codec 226 via a communication link (not shown). The videoand audio processing pipelines output data to an A/V (audio/video) port228 for transmission to the television or other display. In theillustrated implementation, the video and audio processing components220-228 are mounted on the module 214.

Also implemented on the module 214 are a USB host controller 230 and anetwork interface 232. The USB host controller 230 is coupled to the CPU200 and the memory controller 202 via a bus (e.g., PCI bus) and servesas host for the peripheral controllers 104(1)-104(4). The networkinterface 232 provides access to a network (e.g., Internet, homenetwork, etc.) and may be any of a wide variety of various wire orwireless interface components including an Ethernet card, a modem, aBluetooth module, a cable modem, and the like.

The game console 102 has two dual controller support subassemblies240(1) and 240(2), with each subassembly supporting two game controllers104(1)-104(4). A front panel I/O subassembly 242 supports thefunctionality of the power button 112 and the eject button 114, as wellas any LEDs (light emitting diodes) or other indicators exposed on theouter surface of the game console. The subassemblies 240(1), 240(2), and242 are coupled to the module 214 via one or more cable assemblies 244.

Eight memory units 140(1)-140(8) are illustrated as being connectable tothe four controllers 104(1)-104(4), i.e., two memory units for eachcontroller. Each memory unit 140 offers additional storage on whichgames, game parameters, and other data may be stored. When inserted intoa controller, the memory unit 140 can be accessed by the memorycontroller 202.

A system power supply module 250 provides power to the components of thegaming system 100. A fan 252 cools the circuitry within the game console102.

The game console 102 implements a uniform media portal model thatprovides a consistent user interface and navigation hierarchy to moveusers through various entertainment areas. The portal model offers aconvenient way to access content from multiple different mediatypes—game data, audio data, and video data—regardless of the media typeinserted into the portable media drive 106.

To implement the uniform media portal model, a console user interface(UI) application 260 is stored on the hard disk drive 208. When the gameconsole is powered on, various portions of the console application 260are loaded into RAM 206 and/or caches 210, 212 and executed on the CPU200. The console application 260 presents a graphical user interfacethat provides a consistent user experience when navigating to differentmedia types available on the game console.

The gaming system 100 may be operated as a standalone system by simplyconnecting the system to a television or other display. In thisstandalone mode, the gaming system 100 allows one or more players toplay games, watch movies, or listen to music. However, with theintegration of broadband connectivity made available through the networkinterface 232, the gaming system 100 may further be operated as aparticipant in a larger network gaming community. This network gamingenvironment is described next.

FIG. 3 shows an exemplary network gaming environment 300 thatinterconnects multiple gaming systems 100(1), . . . , 100(g) via anetwork 302. The network 302 represents any of a wide variety of datacommunications networks. It may include public portions (e.g., theInternet) as well as private portions (e.g., a residential Local AreaNetwork (LAN)), as well as combinations of public and private portions.Network 302 may be implemented using any one or more of a wide varietyof conventional communications media including both wired and wirelessmedia. Any of a wide variety of communications protocols can be used tocommunicate data via network 302, including both public and proprietaryprotocols. Examples of such protocols include TCP/IP, IPX/SPX, NetBEUI,etc.

In addition to gaming systems 100, one or more online services 304(1), .. . , 304(s) may be accessible via the network 302 to provide variousservices for the participants, such as hosting online games, servingdownloadable music or video files, hosting gaming competitions, servingstreaming audio/video files, and the like. The network gamingenvironment 300 may further involve a key distribution center 306 thatplays a role in authenticating individual players and/or gaming systems100 to one another as well as online services 304. The distributioncenter 306 distributes keys and service tickets to valid participantsthat may then be used to form games amongst multiple players or topurchase services from the online services 304.

The network gaming environment 300 introduces another memory sourceavailable to individual gaming systems 100—online storage. In additionto the portable storage medium 108, the hard disk drive 208, and thememory unit(s) 140, the gaming system 100(1) can also access data filesavailable at remote storage locations via the network 302, asexemplified by remote storage 308 at online service 304(s).

FIG. 4 is a block diagram of another illustrative online gamingenvironment 400, e.g. XBOX® LIVE by Microsoft Corporation of Redmond,Wash. Multiple game consoles 402(1), 402(2), . . . , 402(n) are coupledto a security gateway 404 via a network 406. Each game console 402 canbe, for example, a game console 102 of FIG. 1 or FIG. 2. Network 406represents any one or more of a variety of conventional datacommunications networks. Network 406 will typically include packetswitched networks, but may also include circuit switched networks.Network 406 can include wire and/or wireless portions. In one exemplaryimplementation, network 406 includes the Internet and may optionallyinclude one or more local area networks (LANs) and/or wide area networks(WANs). At least a part of network 406 is a public network, which refersto a network that is publicly-accessible. Virtually anyone can accessthe public network.

In some situations, network 406 includes a LAN (e.g., a home network),with a routing device situated between game console 402 and securitygateway 404. This routing device may perform network address translation(NAT), allowing the multiple devices on the LAN to share the same IPaddress on the Internet, and also operating as a firewall to protect thedevice(s) on the LAN from access by malicious or mischievous users viathe Internet.

Security gateway 404 operates as a gateway between public network 406and a private network 408. Private network 408 can be any of a widevariety of conventional networks, such as a local area network. Privatenetwork 408, as well as other devices discussed in more detail below, iswithin a data center 410 that operates as a secure zone. Data center 410is made up of trusted devices communicating via trusted communications.Thus, encryption and authentication within secure zone 410 is notnecessary. The private nature of network 408 refers to the restrictedaccessibility of network 408—access to network 408 is restricted to onlycertain individuals (e.g., restricted by the owner or operator of datacenter 410).

Security gateway 404 is a cluster of one or more security gatewaycomputing devices. These security gateway computing devices collectivelyimplement security gateway 404. Security gateway 404 may optionallyinclude one or more conventional load balancing devices that operate todirect requests to be handled by the security gateway computing devicesto appropriate ones of those computing devices. This directing or loadbalancing is performed in a manner that attempts to balance the load onthe various security gateway computing devices approximately equally (oralternatively in accordance with some other criteria).

Also within data center 410 are: one or more monitoring servers 412; oneor more presence and notification front doors 414, one or more presenceservers 416, one or more notification servers 418, and a profile store428 (collectively implementing a presence and notification service orsystem 430); one or more match front doors 420 and one or more matchservers 422 (collectively implementing a match service); and one or morestatistics front doors 424 and one or more statistics servers 426(collectively implementing a statistics service). The servers 416, 418,422, and 426 provide services to game consoles 402, and thus can bereferred to as service devices. Other service devices may also beincluded in addition to, and/or in place of, one or more of the servers416, 418, 422, and 426. Additionally, although only one data center isshown in FIG. 4, alternatively multiple data centers may exist withwhich game consoles 402 can communicate. These data centers may operateindependently, or alternatively may operate collectively (e.g., to makeone large data center available to game consoles 102,402).

Game consoles 402 are situated remotely from data center 410, and accessdata center 410 via network 406. A game console 402 desiring tocommunicate with one or more devices in the data center logs in to thedata center and establishes a secure communication channel between theconsole 402 and security gateway 404. Game console 402 and securitygateway 404 encrypt and authenticate data packets being passed back andforth, thereby allowing the data packets to be securely transmittedbetween them without being understood by any other device that maycapture or copy the data packets without breaking the encryption. Eachdata packet communicated from game console 402 to security gateway 404,or from security gateway 404 to game console 402 can have data embeddedtherein. This embedded data is referred to as the content or datacontent of the packet. Additional information may also be inherentlyincluded in the packet based on the packet type (e.g., a heartbeatpacket).

The secure communication channel between a console 402 and securitygateway 404 is based on a security ticket. Console 402 authenticatesitself and the current user(s) of console 402 to a key distributioncenter 428 and obtains, from key distribution center 428, a securityticket. Console 402 then uses this security ticket to establish thesecure communication channel with security gateway 404. In establishingthe secure communication channel with security gateway 404, the gameconsole 402 and security gateway 404 authenticate themselves to oneanother and establish a session security key that is known only to thatparticular game console 402 and the security gateway 404. This sessionsecurity key is used to encrypt data transferred between the gameconsole 402 and the security gateway cluster 404, so no other devices(including other game consoles 402) can read the data. The sessionsecurity key is also used to authenticate a data packet as being fromthe security gateway 404 or game console 402 that the data packetalleges to be from. Thus, using such session security keys, securecommunication channels can be established between the security gateway404 and the various game consoles 402.

Once the secure communication channel is established between a gameconsole 402 and the security gateway 404, encrypted data packets can besecurely transmitted between the two. When the game console 402 desiresto send data to a particular service device in data center 410, the gameconsole 402 encrypts the data and sends it to security gateway 404requesting that it be forwarded to the particular service device(s)targeted by the data packet. Security gateway 404 receives the datapacket and, after authenticating and decrypting the data packet,encapsulates the data content of the packet into another message to besent to the appropriate service via private network 408. Securitygateway 404 determines the appropriate service for the message based onthe requested service(s) targeted by the data packet.

Similarly, when a service device in data center 410 desires tocommunicate data to a game console 402, the data center sends a messageto security gateway 404, via private network 408, including the datacontent to be sent to the game console 402 as well as an indication ofthe particular game console 402 to which the data content is to be sent.Security gateway 404 embeds the data content into a data packet, andthen encrypts the data packet so it can only be decrypted by theparticular game console 402 and also authenticates the data packet asbeing from the security gateway 404.

Although discussed herein as primarily communicating encrypted datapackets between security gateway 404 and a game console 402,alternatively some data packets may be partially encrypted (someportions of the data packets are encrypted while other portions are notencrypted). Which portions of the data packets are encrypted and whichare not can vary based on the desires of the designers of data center410 and/or game consoles 402. For example, the designers may choose toallow voice data to be communicated among consoles 402 so that users ofthe consoles 402 can talk to one another—the designers may furtherchoose to allow the voice data to be unencrypted while any other data inthe packets is encrypted. Additionally, in another alternative, somedata packets may have no portions that are encrypted (that is, theentire data packet is unencrypted). It should be noted that, even if adata packet is unencrypted or only partially encrypted, all of the datapacket can still be authenticated.

Each security gateway device in security gateway 404 is responsible forthe secure communication channel with typically one or more gameconsoles 402, and thus each security gateway device can be viewed asbeing responsible for managing or handling one or more game consoles.The various security gateway devices may be in communication with eachother and communicate messages to one another. For example, a securitygateway device that needs to send a data packet to a game console thatit is not responsible for managing may send a message to all the othersecurity gateway devices with the data to be sent to that game console.This message is received by the security gateway device that isresponsible for managing that game console and sends the appropriatedata to that game console. Alternatively, the security gateway devicesmay be aware of which game consoles are being handled by which securitygateway devices—this may be explicit, such as each security gatewaydevice maintaining a table of game consoles handled by the othersecurity gateway devices, or alternatively implicit, such as determiningwhich security gateway device is responsible for a particular gameconsole based on an identifier of the game console.

Monitoring server(s) 412 operate to inform devices in data center 410 ofan unavailable game console 402 or an unavailable security gatewaydevice of security gateway 404. Game consoles 402 can become unavailablefor a variety of different reasons, such as a hardware or softwarefailure, the console being powered-down without logging out of datacenter 410, the network connection cable to console 402 beingdisconnected from console 402, other network problems (e.g., the LANthat the console 402 is on malfunctioning), etc. Similarly, a securitygateway device of security gateway 404 can become unavailable for avariety of different reasons, such as hardware or software failure, thedevice being powered-down, the network connection cable to the devicebeing disconnected from the device, other network problems, etc.

Each of the security gateway devices in security gateway 404 ismonitored by one or more monitoring servers 412, which detect when oneof the security gateway devices becomes unavailable. In the event asecurity gateway device becomes unavailable, monitoring server 412 sendsa message to each of the other devices in data center 410 (servers,front doors, etc.) that the security gateway device is no longeravailable. Each of the other devices can operate based on thisinformation as it sees fit (e.g., it may assume that particular gameconsoles being managed by the security gateway device are no longer incommunication with data center 410 and perform various clean-upoperations accordingly). Alternatively, only certain devices may receivesuch a message from the monitoring server 412 (e.g., only those devicesthat are concerned with whether security gateway devices are available).

Security gateway 404 monitors the individual game consoles 402 anddetects when one of the game consoles 402 becomes unavailable. Whensecurity gateway 404 detects that a game console is no longer available,security gateway 404 sends a message to monitoring server 412identifying the unavailable game console. In response, monitoring server412 sends a message to each of the other devices in data center 410 (oralternatively only selected devices) that the game console is no longeravailable. Each of the other devices can then operate based on thisinformation as it sees fit.

Presence server(s) 416 hold and process data concerning the status orpresence of a given user logged in to data center 410 for online gaming.Notification server(s) 418 maintains multiple notification queues ofoutgoing messages destined for a player logged in to data center 410.Presence and notification front door 414 is one or more server devicesthat operate as an intermediary between security gateway 404 and servers416 and 418. One or more load balancing devices (not shown) may beincluded in presence and notification front door 414 to balance the loadamong the multiple server devices operating as front door 414. Securitygateway 404 communicates messages for servers 416 and 418 to the frontdoor 414, and the front door 414 identifies which particular server 416or particular server 418 the message is to be communicated to. By usingfront door 414, the actual implementation of servers 416 and 418, suchas which servers are responsible for managing data regarding whichusers, is abstracted from security gateway 404. Security gateway 404 cansimply forward messages that target the presence and notificationservice to presence and notification front door 414 and rely on frontdoor 414 to route the messages to the appropriate one of server(s) 416and server(s) 418.

Match server(s) 422 hold and process data concerning the matching ofonline players to one another. An online user is able to advertise agame available for play along with various characteristics of the game(e.g., the location where a football game will be played, whether a gameis to be played during the day or at night, the user's skill level,etc.). These various characteristics can then be used as a basis tomatch up different online users to play games together. Match front door420 includes one or more server devices (and optionally a load balancingdevice(s)) and operates to abstract match server(s) 422 from securitygateway 404 in a manner analogous to front door 414 abstractingserver(s) 416 and server(s) 418.

Statistics server(s) 426 hold and process data concerning variousstatistics for online games. The specific statistics used can vary basedon the game designer's desires (e.g., the top ten scores or times, aworld ranking for all online players of the game, a list of users whohave found the most items or spent the most time playing, etc.).Statistics front door 426 includes one or more server devices (andoptionally a load balancing device(s)) and operates to abstractstatistics server(s) 426 from security gateway 404 in a manner analogousto front door 414 abstracting server(s) 416 and server(s) 418.

Thus, it can be seen that security gateway 404 operates to shielddevices in the secure zone of data center 410 from the untrusted, publicnetwork 406. Communications within the secure zone of data center 410need not be encrypted, as all devices within data center 410 aretrusted. However, any information to be communicated from a devicewithin data center 410 to a game console 402 passes through securitygateway cluster 404, where it is encrypted in such a manner that it canbe decrypted by only the game console 402 targeted by the information.

Illustrative Embodiments of the Invention

Voice Chat

One or more aspects of the invention may be embodied incomputer-executable instructions (i.e., software), such as in a gameconsole control application software module stored in RAM memory 206,non-volatile memory 108, 208, 308, or any other resident memory on gameconsole 102. Generally, software modules include routines, programs,objects, components, data structures, etc. that perform particular tasksor implement particular abstract data types when executed by a processorin a computer or other device. The computer executable instructions maybe stored on a computer readable medium such as a hard disk 208,removable storage media 108, solid state memory, RAM 206, etc. As willbe appreciated by one of skill in the art, the functionality of thesoftware modules may be combined or distributed as desired in variousembodiments. In addition, the functionality may be embodied in whole orin part in firmware or hardware equivalents such as application specificintegrated circuits (ASIC), field programmable gate arrays (FPGA), andthe like.

The network gaming environment 400 may allow users to invite other usersto participate in a gaming session. Each user of the network gamingenvironment may maintain a ‘Friends list’ identifying players with whomthat user wants to be kept informed when the players are online, as isknown in the art. For example, a user with the online ID (also referredto as a gamertag) Aviator might have a Friends list identifying userswith the gamertags finebokeh, flyerscaps, laBWrats, Doc, and Thrasher.When Aviator is signed in to the network gaming environment 400, thepresence server 416 can inform Aviator when any of finebokeh,flyerscaps, laBWrats, Doc, or Thrasher are also signed in to the networkgaming environment 400. If Aviator wants to play a game, e.g., TOP SPINtennis, with finebokeh from Aviator's Friends list, upon determiningthat finebokeh is also signed in Aviator can send a game invitation tofinebokeh by sending a message through notification server 418.Notification server 418 in turn forwards the message to finebokeh,instructing finebokeh to insert the TOP SPIN game media into gameconsole 402 if not already inserted.

With additional reference to FIGS. 1-8, when a user of a game console100 powers on the game console without a game title in the portablemedia drive 106, the game console loads console application 260providing a default application and user interface for the user tointeract with the game console 100, e.g., to perform console setup(date, time, etc.), music playback, memory management, network setup andmanagement, and the like. According to an aspect of the invention, theconsole application 260 may be adapted to include voice chat support,thus eliminating the need to have the same game media as a Friend inorder to meet up and voice chat to discuss game plans. Voice chat in theconsole application provides a catalyst for building a better onlinecommunity, and allows gamers to quickly and easily get together withFriends in a virtual lobby with the ability to talk to each other. Inone embodiment of the invention, in order to support large communitiesto gather, the voice chat feature may allow up to 16 users, or Friends,to meet together concurrently in a voice chat session. However, more orless than 16 may alternatively be allowed, depending on system needs,capabilities, bandwidth, design choices, etc.

FIG. 5 illustrates a main network menu screen 501 of a consoleapplication according to an illustrative embodiment of the invention.Main network menu screen 501 may include a menu 503 having menu itemsincluding Friends 505, Voice Chat 507, Account Management 509, and MainMenu 511. The ‘A’ button on game controller 104 may be used to select acurrently highlighted menu item, whereas the ‘B’ button on gamecontroller 104 may be used to navigate back up the menu hierarchy ofgame console 100, as illustrated by control indicators 515, 517. Themain network menu screen 501 may include other information, such as thenumber of Friends online, outstanding game invitations, and the like.

Upon selecting the Voice Chat option 507, the console application mayask the user for an account passcode (not shown) to verify that the useris the owner of the account being accessed. Upon successful completionof the passcode challenge, the console application retrieves the user'sFriends list and queries the presence and notification servers 416 and418, respectively, to determine which Friends 505, if any, areconcurrently signed in to the online gaming environment 400.

With reference to FIG. 6, console application 260 displays out-of-gamevoice chat screen 601 including voice chat list 603. An out-of-gamevoice chat, as used herein, refers to a voice chat that executes from anon-removable memory associated with game console 102, and that iscapable of executing independently of a computer or video game that canexecute on game console 102. While the inventors recognize thatvirtually any memory can be removed with enough effort, by non-removablethe inventors refer to a memory not used by default to store primarygame media files (i.e., it may store save game or state info, but is notused as primary storage for game executables). Examples of non-removablememory may include a hard disk drive. Examples of removable memory mayinclude CD or DVD drives.

Voice chat list 603 displays all gamers currently in the voice chatsession. The console application may display a Friends button 605 at thetop of the voice chat list which, upon selection by the user, causes theconsole application to display the user's Friends list. The user canselect a Friend from his or her Friends list, which causes the consoleapplication to bring up a contextual menu with an option to “Invite toChat,” as illustrated in FIG. 7. Upon selecting the “Invite to Chat,”the console application sends a chat invitation to the selected Friend.The selected Friend then receives the invitation voice chat. The invitedFriend can accept the voice chat invite and the console applicationnavigates the invited Friend to the voice chat session area of his orher own game console's console application. The voice chat screen 601may optionally include additional information such as player statistics,rankings, etc.

Once in a voice chat session with a Friend or Friends, players may useheadset 142 to talk to each other in real-time voice. If a gamer doesnot have a speaker/microphone appropriately connected (e.g., the headsetconnected to a game controller), the voice from others in the voice chatsession may be played through the television or stereo speakers. In thisway, gamers are still able to be a part of the voice chat community eventhough they cannot themselves speak to the other players in the voicechat session. Additionally, this allows multiple persons in the samephysical room to listen.

While the invention has been thoroughly described above, a more detailedmethod for establishing and administering a voice chat session will nowbe described with reference to FIG. 8. The method of claim 8 starts whena user decides he or she wants to initiate a voice chat with anotheruser. The initiating user, referred to herein as the host, creates avoice chat session in step 805. The host can initiate a voice chatsession in a variety of ways. First, the host can select the voice chatoption 507 (FIG. 5) from the main network menu screen 501. Uponselecting the voice chat option, and entering his or her pass code ifnot already signed in to the online gaming environment, the host'sconsole 100 sends a voice chat session request to online service 304,e.g., to data center 410. Data center 410 records a host ID, creates anew voice chat session ID, and sends the session ID for the voice chatsession to host.

Alternatively, the host may select Friends option 505 from menu screen501. Selecting Friends option 505 causes the console to retrieve andupdate the host's Friends list, including online/offline status of userson the Friends list, and outstanding Friend invitations, and outstandinggame invitations. The host, being able to see which Friends (if any) arealso online, can select a Friend from on the Friends list. Uponselecting the Friend, the console may provide the host with a number ofoptions to perform based on the selected Friend, one of which may be‘Voice Chat’. Upon selecting the Voice Chat option the host's console100 sends the voice chat session request to online service 304, e.g., todata center 410. Data center 410 records a host ID, creates a new voicechat session ID, and sends the session ID for the voice chat session tohost.

Once the voice chat session has been created, the host in step 810 caninvite other users to participate in the voice chat. In the secondscenario discussed above, where the host selects a Friend to chat withfrom the Friends list options menu, the invitation may be automaticallysent to the selected Friend without further input from the host. In theother scenario, where the host selects Voice Chat from the menu 503 onnetwork screen 501, the host manually selects additional users to inviteto the voice chat by selecting the Friends button 605 on voice chatscreen 601, causing the host's Friends list to be displayed, similar toselecting Friends option 505 from network screen 501. The host cansubsequently select a Friend or Friends to which additional invitationswill be sent.

In some embodiments of online gaming environment 400, each invitationpreferably includes at least a session ID, and a title ID identifyingthe game title to which the invitation pertains. The invitation may alsoinclude inviting user and invitee data. For example, for a situationwhere GAMERTAG002 sends an invitation to GAMERTAG005 to play the gameCRIMSON SKIES®, the invitation would preferably include a session ID, atitle ID (e.g., information relating to the game CRIMSON SKIES®), the IDof the inviting user (e.g., GAMERTAG002), and the ID of the invitee(e.g., GAMERTAG005). In the case where a user wants to have anout-of-game voice chat with another user, in order to maintainconsistency and user compatibility with the game play relatedinvitations, each invitation would preferably still have a session ID,and inviting user and invitee data. It also preferably has a title ID,except the title ID would not corresponding to the name and data of aspecific game but would instead correspond to an out-of-game voice chatapplication resident on the game console. Accordingly, such a title IDwould not have a “game title” ID but would designate details relating tothe out-of-game chat application. Thus, for a situation whereGAMERTAG002 sends an invitation to GAMERTAG005 to voice chat, theinvitation would preferably include a session ID, a title ID equivalentto an out-of-game voice chat application resident on the game console,the ID of the inviting user (e.g., GAMERTAG002), and the ID of theinvitee (e.g., GAMERTAG005). In some embodiments, the invitation mayalso include a recorded voice message from the voice chat host, e.g.,describing a subject matter of the voice chat, a personal greeting, orthe like. The recorded voice message may be any length depending onsystem capabilities. In one embodiment, in order to conserve resources,up to a fifteen second recorded message may be used. Console 100 thensends the invitation to data center 410 for transmission to the inviteevia presence and notification servers 416, 418, respectively.

Upon receiving an invitation, an invitee can decide to accept theinvitation in step 815. If not already signed in to the online gamingenvironment 400, the invitee may be required to perform security checksand authentication in step 820. Security and authentication may includeperforming a key exchange with key distribution center 428. Once aninvitee has completed security and authentication, invitee obtains thehost's IP address in step 825 from data center 410. The invitee sends anintroduction message to the host in step 830, and obtains the IPaddresses of all voice chat participants. The host, likewise, notes theIP address for each participant from which it receives an introductionmessage.

Once the host and each invitee have the IP addresses for all voice chatparticipants, the voice chat commences in step 835. During the voicechat, each user's voice is compressed using any known voice or audiocodec. For example, a voice or audio codec from Nellymoser, Inc. ofArlington, Mass. may be used, or a Windows Media 9 voice or audio codec,available from Microsoft Corporation of Redmond, Wash., mayalternatively be used. One of skill in the art will appreciate that anynow known or later developed voice and/or audio codec may be used tosuit the needs of the specific voice chat implementation. The compressedvoice data may then be sent encrypted or unencrypted to all voice chatparticipants, based on the known IP addresses of the participants. Thatis, because IP information is exchanged between all voice chatparticipants, the voice chat participants are connected as a sort ofinterconnected mesh, independent of a voice chat server, and cancommunicate directly with each other.

A voice chat participant can terminate or exit the voice chat in step840. A non-host participant, upon selecting the back (B) option 611, maybe automatically terminated from the voice chat session or may be giventhe option to confirm exiting or staying in the voice chat. When thehost user selects Back option 611, the data center removes the sessioninformation for that voice chat session and the voice chat session maybe terminated for all voice chat participants (because the host was theregistered user with data center 410 for that particular voice chatsession).

Alternatively, in some embodiments, the voice chat session may continueeven after the original host exited the voice chat. This may beaccomplished using a host-migration chat model. For example, the datacenter 410 may keep track of multiple voice chat participant IPaddresses per voice chat session, e.g., a primary and backup host,instead of merely the session host. If the primary host exits the voicechat session, the data center 410 may move the backup host to theprimary host slot, and select a new backup host from the remaining voicechat participants (e.g., by requesting the new host to identify an IPaddress of a remaining participant). If the backup host exits the voicechat session, the data center 410 may simply select a new backup hostfrom the remaining voice chat participants.

While the above description provides details for a voice chat in aconsole application, in some embodiments the voice chat may incorporateone ore more usability and/or special features to enhance the voice chatexperience for voice chat participants. For example, the consoleapplication may use sound effects to reinforce when a player has joinedor left the chat session or signed in or out of the online gamingenvironment; the ‘Host’ (i.e., the first person in the chat session) mayhave the ability to select a gamer in the voice chat session and changethe user's voice chat settings (e.g., kick him or her from the voicechat session if the host feels it necessary to do so, or change thevoice masking effect currently applied to that user, if any); and theconsole application may use visual feedback to identify a currentspeaker by flashing a speaker icon 607 next to the player currentlyspeaking in the voice chat session, or provide a Friends icon 609 toidentify players currently on a player's own Friends list. Friends icon609 may be used because there may be non-friends on the list who enteredthe chat via a shared friend. For example, A and B are friends, and Band C are friends. A invites B to the chat, and B then invites C to thechat. A, B and C are all then in the chat, but A and C are not presentlyfriends. A can send C a friend request by selecting C on the voice chatscreen and selecting an “add to friends list” or similar option.

For example, in the example illustrated in FIG. 6, Aviator is in a chatsessions with users with gamertags GAMERTAG001, GAMERTAG002,GAMERTAG003, GAMERTAG004, GAMERTAG005, and GAMERTAG006. Furthermore,FIG. 6 illustrates that GAMERTAG001 and GAMERTAG004 are both currentlytalking, and that all users participating in the chat session except forGAMERTAG003 and GAMERTAG004 are currently on Aviator's Friends list. Allplayers in the voice chat session may have the ability to select othergamers in the session to either add or remove them from their ownFriends list. Aviator can thus add GAMERTAG003 and GAMERTAG004 to hisfriends list if desired. In this manner, a gamer's circle of Friends canquickly grow to encircle his or her Friends' Friends who are able tojoin the chat session as well.

In some embodiments, where each voice chat session is registered withmatch server 422 and a voice chat session host has designated thesession as being open to non-Friend participants, users may be ablesearch for a chat session based on various criteria and/or user profileinformation. For example, a user may search for an open voice chatsession based on ranking or skill level of voice chat participants (orsimply the host), games or game types played, game attitude (e.g., hardcore, spare time player, socialite, etc.), or any other informationtracked by the online game environment 400.

In some embodiments the host (either exclusively or nonexclusively ofother voice chat participants) can initiate a vote in the voice chatsession, e.g., by selecting options button 613 and selecting ‘CreateVote’ (not shown). The host can enter a question with a known answerdomain (e.g., yes/no; true/false; A/B/C/D, etc.) and submit the questionfor a vote. Each voice chat participant, optionally including the host,then responds to the question by manipulating the controls on controller104 (e.g., move joystick up for yes/true/A; move joystick down forno/false/C; move joystick right for B, move joystick left for D). Inthis manner voice chat participants can vote yes/no for whether acertain action should be taken (e.g., whether to play a game proposed bythe host; true/false for responding to appropriate vote questions; orselect an A, B, C, or D answer from a proposed set of answers such aswhich of four proposed games to play together). Other options (notshown) that may be available upon selection of option button 613 includean option to appear offline to non-voice chat participants when in avoice chat session and to force voice chat sound through the TVspeakers, regardless of whether a microphone is connected.

In some embodiments there may be a whisper mode (also referred to as‘secret’ mode) whereby a voice chat participant can optionally speak toonly a designated other voice chat participant, even when there are morethan two voice chat participants in a given voice chat session. In suchan embodiment, a user may select a voice chat participant from list 603,and one of the resulting options may be ‘Whisper’. Upon selectingWhisper, the voice chat screen 601 may be modified slightly to indicatethat the user's subsequent speech will only be sent to the selected useruntil such time as the user exits the whisper mode, e.g., by selectingthe back button (B) or by releasing a button that was otherwise used toenter the whisper mode. In some embodiments, the user may configure acontrol input on the controller 104 such that whenever the user isactivating the selected control input, voice data will only be sent to aspecified user corresponding to that control input. In some embodiments,the user may set up multiple whisper modes, e.g., whisper to a firstuser when holding the ‘X’ button, whisper to a second user when holdingthe ‘Y’ button, whisper to a third user when holding the left trigger,etc.

Similar controls may be set up to control voice masking effects usedduring the voice chat. That is, a user might configure their voice chatsettings to indicate that, by default, no voice masking should be used,but that a cheerleader voicemask should be used while the thumbpad ispressed the left, a robot voicemask should be used while the thumbpad ispressed to the right, etc. The controls used here are illustrative only,and one of ordinary skill in the art will appreciate that whisper andvoicemask settings may be mapped to any otherwise unused control inputor combination of control inputs on controller 104.

In some embodiments voice chat may include a video chat option. In suchan embodiment, a user need only have a video camera peripheral connectedto game console 102 and have a video chat option selected. In such ascenario, video may be compressed using a video codec and sent to theother voice chat participants similar to compressed voice data.

According to an aspect of the invention, voice chat participants may beable to share files during voice chat. That is, when game console 102 isable to store files on hard disk drive 208, e.g., music files, picturefiles, saved game files, etc, a voice chat participant desiring to senda file to another voice chat participant may select a desired recipientfrom list 603 and select a ‘Send File’ option (not shown). The sendinguser then navigates through a hierarchical menu or file system to locatethe desired file to send to the other user. Upon selecting a file, thegame console 102 sends the selected file to the selected user based onthe known IP address of the selected user, e.g., using TCP/IP, FTP, etc.A reciprocal “Send to User” command may appear on an options menu whennavigating a file system external to the voice chat. In someembodiments, a user may specify a music file for playback during thevoice chat session, whereby the game console both plays back theselected audio file and sends the audio file to the voice chatparticipants, whose game consoles automatically play the transmittedaudio file as background music to the voice chat.

Emoticons

In some embodiments of the invention, a participant in a voice chatsession can send an emoticon to another participant in the voice chat.As used herein, the term emoticon is broadly defined as a conveyance ofemotion or action in a chat session from one participant to another overthe network in a manner using neither the primary means of communicationof the chat session nor game-play features (when chat is in-game). Forexample, in a voice chat, an emoticon may include virtually anythingsent over the network, other than one participant speaking directly tothe one or more other voice chat participants. If a chat session isin-game, a player-character performing an action in-game that can beotherwise performed during game play is not considered an emoticon.

Gaming system environment 100 includes A/V interfacing cables 120 foroutputting visual display elements to video and auditory elements to oneor more speakers. Gaming system environment 100 also includes one oremore controllers 104, which may further incorporate force feedbackcapabilities. Force feedback, generally, refers to the ability of acontroller to exert positive force, e.g., by vibrating or rumbling, orby actively exerting pressure on a control input such as a joystick,flight yoke, or driving wheel. Using these capabilities of gaming systemenvironment 100, a voice chat session can provide dynamic emoticoncapabilities that encompass the haptic (touch), visual (sight), and/orauditory (hearing) senses, thus enabling one participant to physicallyaffect another participant's gaming system environment.

Haptic emoticons affect the sense of touch. For example, hapticemoticons may include rumbling or vibrating another voice chatparticipant's controller 104, and causing another voice chatparticipant's joystick, flight yoke, or driving wheel to physically movewithout receiving input from the other voice chat participant.

A visual emoticon can be anything that causes another voice chatparticipant's display to be altered. Examples of visual emoticons areendless. For example, there may be visual effects emoticons that affectanother voice chat partipant's display. A ‘negative’ emoticon may causeanother voice chat participant's display to switch to a negative image;a ‘melt’ emoticon may cause another voice chat participant's display toappear as if it is melting off the screen; and a ‘blur’ emoticon maycause another voice chat participant's display to appear blurry andunfocused so that the other voice chat participant cannot easily focusor make menu selections. These are merely three examples of possiblevisual effects emoticons. Visual emoticons may also include moretraditional icons, graphics or animations displayed on the voice chatscreen 601 (FIG. 6), either on the screen in general, or next to thevoice chat participant sending the emoticon. For example, if GamerTag003sends a smiley face emoticon, e.g., :), :-) or

, the emoticon may be displayed next to GamerTag003 on list 603; if theemoticon includes an animation of a fist pounding the screen, theemoticon may be displayed on a larger portion of the view screen.

Auditory emoticons are likewise endless in possibility. An auditoryemoticon is one which causes sound to be output via anotherparticipant's gaming system 102, or one which affects sound output viaanother participant's gaming system 102, and may be desirable to summona participant who has not spoken in a voice chat session for some time,or to simply show emotion of give commands to a fellow voice chatter.Auditory emoticons may be as simple as one voice chat participantcausing a “beep” to be output through the gaming system 102 of anothervoice chat participant, or may initiate playback of a more complex orsynthesized sound, such as a fog horn, or prerecorded sounds, such asmusic, speech, or the like. An auditory emoticon may also affect soundotherwise output through another participant's gaming system 102. Forexample, an ‘echo’ auditory emoticon may cause all sound output via theselected participant's gaming system 102 to have an echo sound effect; a‘radio noise’ auditory emoticon may cause all sound output via theselected participant's gaming system 102 to have static similar to a badradio signal; a ‘cavern’ auditory emoticon may cause all sound outputvia the selected participant's gaming system 102 to sound as if thespeaker were in a large cavern or other large room. Auditory emoticonsthat affect sound may be applied by emoticon recipient game console 102using a selected sound effect filter for the duration of the auditoryemoticon.

Any combination of haptic, visual, and auditory elements may be combinedin a single emoticon, referred to herein as a combination emoticon. Anexample of a combination emoticon imitates an ‘earthquake’ emoticon thatcauses the controller to rumble or vibrate (haptic), causes the displayto blur or shake violently (visual), and plays a sound of a largestructure collapsing (auditory). Another example of a combination iconimitates an electromagnetic pulse (EMP) by applying a degaussing effecton the video display and outputting appropriate electrical sounds viathe speakers. Other examples might include images with added animatedvisual and/or audio effects that appear on screen for the selectedrecipient(s), such as an animated smiley face or a rock fist that poundsthe screen, shaking all content presently onscreen. These are but a fewexamples of combination emoticons, and those of skill in the art willappreciate that endless possibilities of combination emoticons may besubsequently designed.

A voice chat participant may send an emoticon to another voice chatparticipant in various ways, depending on the design and user interfaceof the particular network gaming environment in which voice chat andemoticons are implemented. In one illustrative embodiment, withreference to FIG. 6 and FIG. 9, when a voice chat participant selectsanother voice chat participant from voice chat screen 601 (FIG. 6), forexample selecting the voice chat participant with gamertag GamerTag001,the game console 102 may display a menu 901 (FIG. 9) displaying optionswith respect to the selected voice chat participant. As discussed above,options may include inviting the selected voice chat participant to jointhe selecting participant's Friends list (or removing the selected voicechat participant from the selecting participant's Friends list, ifalready on it), removing the selected voice chat participant from thevoice chat session (this option might appear only to the host of thevoice chat session), and sending a whisper or private chat to a user.Another option that may be displayed is an option 902 to send anemoticon to the selected voice chat participant.

Upon selection of emoticon option 902, the game console may display anemoticon submenu 1001, such as that illustrated in FIG. 10, thatprovides a list of emoticons that may be sent from one voice chatparticipant to another. FIG. 10 illustrates an emoticon submenu in anillustrative embodiment where a voice chat participant may select fromany of four emoticons to send to another voice chat participant,including a haptic tickle emoticon 1003, a visual screen negativeemoticon 1005, an auditory fog horn emoticon 1007, and a combinationearthquake emoticon 1009.

Upon selecting the desired emoticon, the selecting voice chatparticipant's game console 102 sends emoticon information (discussedbelow) to the game console of the selected voice chat participant,resulting in the recipient's game console performing the requestedemoticon. When the recipient's game console performs the receivedemoticon, the recipient's game console may simultaneously display anindicator in list 603 (FIG. 6) next to the gamertag of the emoticonsending voice chat participant, highlight the sender, or provide someother indication of the emoticon sending voice chat participant.

In one illustrative embodiment, the emoticon execution may occur for aslong as the selecting voice chat participant continues pressing theselect button ‘A,’ optionally up to a maximum length of time. In anotherembodiment, the emoticon may be instance based, occurring only once fora predetermined length of time each time the select button ‘A’ ispressed.

According to an aspect of the invention, the haptic, auditory, visual,and/or combination emoticons may have an associated intensity level. Inone illustrative embodiment, where buttons 136 on controller 104 arepressure sensitive, the intensity level may be based on the pressurewith which the button is pressed. Other mechanisms for specifying anintensity level may alternatively be used, such as using incrementalintensity by pressing a button once for low intensity, twice for mediumintensity, and three times for high intensity.

As shown in FIG. 10, the selecting voice chat participant may use button‘A’ to select the tickle emoticon 1003. If the ‘A’ button on controller104 is pressure sensitive, the resulting vibration intensity maycorrespond to the pressure with which the voice chat participant pressesthe ‘A’ button. Softer pressure results in lower resulting vibrationintensity; harder pressure results in stronger resulting vibrationintensity. An example of a pressure sensitive auditory emoticon may be a‘train whistle’ emoticon, where the train whistle grows louder as thepressure is increased on the selecting button ‘A’. An example of apressure sensitive visual emoticon may be a ‘blur’ emoticon where theselected voice chat participant's display grows blurrier as the pressureis increased on the button.

Similarly, combination emoticons may also have pressure sensitiveelements across their haptic, auditory, and or visual aspects. Forexample, the ‘earthquake’ emoticon may rumble stronger, shake screenmore violently, and play the crashing sound effect louder as thepressure is increased on the button.

Each game console 102 participating in a particular voice chat sessionneed only know the range of emoticons available in order to provideemoticon services to voice/video chat sessions. Each game console may bestatically programmed with a known list of emoticons. For example, asillustrated in FIG. 10, there are four possible emoticons. More or lessemoticons may be provided. Where more emoticons are available, gameconsole 102 may use hierarchical submenus to allow a voice chatparticipant to navigate to a desired emoticon, such as is illustrated inFIG. 11, or may provide a virtual onscreen keyboard or menu from which auser can select an emoticon using controller 104. Alternatively, wherefewer emoticons are available, they may be placed directly (not shown)on menu 901. An identifier identifying the selected emoticon issubsequently sent from the sender's game console to the recipient's gameconsole, as described below.

FIG. 12 shows a flowchart for an illustrative method to send a dynamicemoticon from a sending voice chat participant (User A) to a recipientvoice chat participant (User B). Initially, in step 1201, User A selectsemoticon recipient User B, e.g., the participant with gamertagGamerTag001, from voice chat screen 601 (FIG. 6). Next, in step 1203,User A selects the emoticon to send to User B by selecting the emoticonfrom one or more menus (see, e.g., FIGS. 9-11, and above descriptionthereof) displayed as a result of selecting a voice chat participantfrom screen 601. In step 1205, User A's game console (GC-A) looks up theIP address of the game console (GC-B) corresponding to User B. GC-A canlook up the IP address, e.g., in a lookup table stored in its own memorybased on the IP address information received as part of the voice chatsession (described above). Those of skill in the art will appreciatethat other address and/or lookup mechanisms may alternatively be used,as are known in the art.

In step 1207 GC-A composes an emoticon instruction message and sends itvia one or more data packets (e.g., using TCP/IP) over network 302and/or network 406 to GC-B corresponding to User B. Each emoticoninstruction message includes, at a minimum, sender identification (e.g.,gamertag, IP address, etc.), recipient identification, and an emoticonidentifier (e.g., 1=Tickle, 2=FogHorn, 3=Earthquake, etc.; “Tickle,”“Foghorn,” “Earthquake,” etc.) based on the universe of emoticons commonto all game consoles in the networked gaming environment. If theemoticon is intensity-based, then the emoticon instruction message mayfurther include an intensity level. For combination emoticons, theemoticon instruction message may include an intensity level (e.g., 1, 2,3, . . . ) for each channel (haptic, auditory, visual) of the emoticonthat is pressure sensitive, or it may include a single intensity levelfor all channels. For continuous emoticons that occur as long as theselect button remains depressed, GC-A may continuously stream emoticoninstruction messages to GC-B until such time as the selected button isreleased, or the emoticon instruction message may include a duration.Alternatively, GC-A may send a START continuous emoticon message withthe requisite details to GC-B. When the selected button is released GC-Amay send a STOP continuous emoticon message to GC-B. Upon receipt of theemoticon instruction message in step 1209, GC-B performs the selectedemoticon identified and/or defined in the emoticon instruction message,and displays the emoticon indicator on screen 601 indicating the voicechat participant who sent the emoticon.

FIG. 13 illustrates a sample emoticon instruction message 1301 accordingto an illustrative embodiment of the invention. Message 1301 includes asender data 1303, recipient data 1305, and emoticon identifier data1307. Message 1301 may optionally include, shown in broken lines,intensity level data 1309 and duration data 1311. FIG. 14 illustrates analternative sample emoticon instruction message 1401 where separateintensity level data is provided for each component of the emoticon.Intensity data 1403 provides an intensity level for the auditorycomponent; intensity data 1405 provides an intensity level for thevisual component, and intensity data 1407 provides an intensity levelfor the haptic component.

The above method and system are one illustrative embodiment of howdynamic emoticons can be performed. Those of skill in the art willappreciate that modifications and changes may be made while still withinthe spirit and scope of the invention. For example, in a possiblealternative embodiment, while in a voice chat session a button 136(e.g., ‘Y’), trigger (e.g., left trigger), or other control input may bepermanently or temporarily mapped to an emoticon or an emoticon mode. Ifmapped to an emoticon, e.g., Tickle, whenever the mapped control inputis pressed or actuated by a voice chat participant, that voice chatparticipant's game console 102 sends a tickle emoticon message to thegame console corresponding to the presently selected voice chatrecipient on voice chat screen 601. The tickle command may further havean intensity based on the button pressure. If mapped to an emoticonmode, whenever the mapped control input (e.g., left trigger) is pressedor actuated by a voice chat participant, the voice chat participant maysimultaneously actuate another control input to send an emoticon mappedto that other control input. When two emoticons are sent in successionso that the receiving game console has not finished executing the firstemoticon when the second emoticon is received, the receiving gameconsole may fade the first emoticon into the second successive emoticon,or in the case an emoticon that has a defined period, the receiving gameconsole may wait for the first emoticon to finish execution beforeexecuting the second emoticon. If the two emoticons do not compete forthe same channel, the receiving game console may execute both emoticonssimultaneously. If two emoticons are competing for the same channel(i.e., audio, visual, haptic) the receiving game console may overridethe channel with the most recently received emoticon. Any combination oralternative of the above resolution methodologies, or other resolutionmethodologies, may be used.

In another alternative embodiment, a voice chat participant may send acustom emoticon to another voice chat participant, e.g., by sending agraphic or audio file stored on the hard drive of the sending voice chatparticipant's game console. In such an embodiment, when the user selects‘custom emoticon,’ his or her game console displays a file navigationmenu to allow the user to select the file containing the audio orgraphic to send to the recipient, similar to a file transfer, exceptthat the graphic or audio file is not saved on the recipient's systemafter it has been output via the display and/or speakers accordingly.

In another alternative embodiment, custom emoticons may include scriptthat instructs a recipient's game console to execute an emoticonaccording to the provided instructions. In such an embodiment, thescript language (known to each game console) may define basic elementsof emoticons (e.g., haptic, auditory, and visual channels), and actionsthat can be performed (e.g., haptic actions such as vibrate and forcefeedback, visual actions such as displaying icons and performing screeneffects, and auditory actions such as playing sounds and implementingsound effects), and users can create custom emoticons by combiningelements and actions as desired.

In yet another alternative embodiment, additional emoticon elements maybe provided in addition to haptic, visual, and auditory. For example, anemoticon may include a system component that controls non-hapticphysical characteristics of the emoticon recipient's system. Theemoticon may cause one or more LEDs on the recipient's system to blinkor flash to signify winking, may eject a media drive tray to signifysticking one's tongue out, and the like. Any electronically controllableaspect of a data processing system may be subject to emoticon control,depending on security and design features of the given system.

Dynamic emoticons as described herein expand on the known emoticon spaceby adding new dimensions and capabilities for emoticons in a chatsession. Dynamic emoticons allow users to send combinations of sound,video, and haptic feedback to other user's televisions, stereos, andgame controllers, thus ensuring an innovative, fun and entertaininggaming environment.

While the invention has been described with respect to specific examplesincluding presently preferred modes of carrying out the invention, thoseskilled in the art will appreciate that there are numerous variationsand permutations of the above described systems and techniques. Thus,the spirit and scope of the invention should be construed broadly as setforth in the appended claims.

1. At least one data packet communicating an emoticon instructionmessage comprising a haptic component.
 2. The at least one data packetcommunicating an emoticon message of claim 1, further comprising anintensity level.
 3. The at least one data packet communicating anemoticon message of claim 2, wherein the intensity level is based on abutton pressure.
 4. The at least one data packet communicating anemoticon message of claim 1, further comprising an auditory component.5. The at least one data packet communicating an emoticon message ofclaim 4, wherein the auditory component identifies a prerecorded file.6. The at least one data packet communicating an emoticon message ofclaim 4, wherein the auditory component identifies a sound effectsfilter.
 7. The at least one data packet communicating an emoticonmessage of claim 4, wherein the auditory component comprises an audiofile for playback on a receiving game console, without permanentlysaving the audio file on the receiving game console.
 8. The at least onedata packet communicating an emoticon message of claim 1, furthercomprising a visual component.
 9. The at least one data packetcommunicating an emoticon message of claim 8, wherein the visualcomponent identifies a prerecorded file.
 10. The at least one datapacket communicating an emoticon message of claim 8, wherein the visualcomponent identifies a visual effects filter for modifying existingcontent displayed on a display device.
 11. The at least one data packetcommunicating an emoticon message of claim 8, wherein the visualcomponent comprises a graphic file for display on a receiving gameconsole, without permanently saving the graphic file on the receivinggame console.
 12. The at least one data packet communicating an emoticonmessage of claim 1, further comprising a duration.
 13. At least one datapacket communicating an emoticon message comprising an intensity level.14. The at least one data packet communicating an emoticon message ofclaim 13, wherein the intensity level is based on a button pressure. 15.At least one data packet communicating an emoticon message comprising anemoticon script.
 16. The at least one data packet communicating anemoticon message of claim 15, wherein the emoticon script defines scriptelements and element actions.
 17. The at least one data packetcommunicating an emoticon message of claim 16, wherein the scriptelements comprise haptic, auditory and visual elements.
 18. At least onedata packet communicating an emoticon message comprising an auditorycomponent.
 19. The at least one data packet communicating an emoticonmessage of claim 18, wherein the auditory component identifies aprerecorded file.
 20. The at least one data packet communicating anemoticon message of claim 18, wherein the auditory component identifiesa sound effects filter.
 21. The at least one data packet communicatingan emoticon message of claim 18, wherein the auditory componentcomprises an audio file for playback on a receiving game console,without permanently saving the audio file on the receiving game console.22. At least one data packet communicating an emoticon messagecomprising a system component.
 23. The at least one data packetcommunicating an emoticon message of claim 22, wherein the systemcomponent alters an on/off state of a physical light element on a dataprocessing system.
 24. The at least one data packet communicating anemoticon message of claim 22, wherein the system component ejects adrive tray on a data processing system.
 25. A gaming system, comprising:a data processing unit for executing entertainment applications from amedia drive, and for executing a voice chat application from a memory; acontroller connected to the data processing unit, the controllercomprising a plurality of controls for providing input to the dataprocessing unit; a network port for communicating with other gamingsystems over a network; the memory storing the voice chat application ascomputer executable instructions, said instructions comprisinggenerating an emoticon message based on user input received from thecontroller.
 26. The gaming system of claim 25, wherein the emoticonmessage comprises a haptic component.
 27. The gaming system of claim 25,wherein the emoticon message comprises an intensity level.
 28. Thegaming system of claim 27, wherein the intensity level is based on abutton pressure component of the received user input.
 29. The gamingsystem of claim 25, wherein the emoticon message comprises an auditorycomponent.
 30. The gaming system of claim 29, wherein the auditorycomponent identifies a prerecorded file.
 31. The gaming system of claim29, wherein the auditory component identifies a sound effects filter.32. The gaming system of claim 29, wherein the auditory componentcomprises an audio file for playback on a receiving game console,without permanently saving the audio file on the receiving game console.33. The gaming system of claim 25, wherein the emoticon messagecomprises a visual component.
 34. The gaming system of claim 33, whereinthe visual component identifies a prerecorded file.
 35. The gamingsystem of claim 33, wherein the visual component identifies a visualeffects filter.
 36. The gaming system of claim 33, wherein the visualcomponent comprises a graphic file for display on a receiving gameconsole, without permanently saving the graphic file on the receivinggame console.
 37. The gaming system of claim 25, wherein the emoticonmessage comprises a duration.
 38. The gaming system of claim 25, whereinthe emoticon message comprises an emoticon script.
 39. The gaming systemof claim 38, wherein the emoticon script defines script elements andelement actions.
 40. The gaming system of claim 39, wherein the scriptelements comprise haptic, auditory and visual elements.
 41. The gamingsystem of claim 25, wherein the emoticon message comprises a systemcomponent.
 42. The gaming system of claim 41, wherein the systemcomponent alters an on/off state of a physical light element on anemoticon recipient's game console.
 43. The gaming system of claim 41,wherein the system component ejects a drive tray on an emoticonrecipient's game console.
 44. The gaming system of claim 41, wherein thememory comprises a resident memory of the gaming system.
 45. A methodfor sending a dynamic emoticon from a game console executing a chatapplication, comprising: receiving first user input identifying thedynamic emoticon; receiving second user input identifying a recipient ofthe dynamic emoticon; composing an emoticon instruction messagecomprising sender data, recipient data based on first user input, and aemoticon data based on second user input; sending the emoticon messageto a second game console corresponding to the identified recipient forplayback on the second game console based on the data in the emoticoninstruction message
 46. The method of claim 45, wherein the emoticoninstruction message comprises a haptic emoticon component.
 47. Themethod of claim 45, wherein the emoticon instruction message comprisesan intensity level.
 48. The method of claim 47, wherein the intensitylevel is based on a button pressure with which the user provided theuser input identifying the dynamic emoticon.
 49. The method of claim 45,wherein the emoticon instruction message comprises an auditorycomponent.
 50. The method of claim 45, wherein emoticon instructionmessage further comprises an identifier identifying a file stored on thesecond game console.
 51. The method of claim 50, wherein the identifieridentifies a prerecorded audio file stored on the second game console.52. The method of claim 50, wherein the identifier identifies an audiofilter stored on the second game console.
 53. The method of claim 45,wherein the emoticon instruction message comprises a visual component.54. The method of claim 50, wherein the identifier identifies a visualeffects filter for modifying existing content displayed by the secondgame cosnole.
 55. The method of claim 45, wherein the emoticoninstruction message comprises a duration.
 56. The method of claim 45,wherein the emoticon instruction message comprises an emoticon script.57. The method of claim 56, wherein the emoticon script defines scriptelements and element actions.
 58. The method of claim 57, wherein thescript elements comprise haptic, auditory and visual elements.
 59. Themethod of claim 45, wherein the emoticon instruction message comprises asystem component.
 60. The method of claim 59, wherein the systemcomponent instructs the second game console to alter an on/off state ofa physical light element on the second game console.